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A stationary  sequence  of  random  variables  with  exponential 
marginal  distributions  and  the  correlation  structure  of  an  ARMA 
(1,1)  process  is  defined.  The  process  is  formed  as  a random 
linear  combination  of  i.i.d.  exponential  random  variables  and 
is  very  simple  to  generate  on  a computer.  Moments  and  joint 
distributions  for  the  sequence  are  obtained,  as  well  as  limiting 
properties  of  sums  of  the  random  variables  and  of  the  point  ^ 
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1.  INTRODUCTION 


We  discuss  the  stationary  sequence  of  random  variables 
(Xn>  which  is  formed  from  a sequence  {*n)  of  independent 
random  variables,  each  with  the  same  exponential  distribution 
with  parameter  X > 0,  according  to  the  probabilistic  linear 
model 


Xn  * | 

l6,n 

ls<n+An-l 

with  probability 
with  probability 

e, 

(l-B)  , 

(1.1) 

for 

n * 1,2, . 

. . , where 

0 s £ a 1 , and 

A„*  | 

1 °Vl 

' pAn-l  * C n 

with  probability 
with  probability 

P# 

(1-p)  , 

(1.2) 

for 

n * 1,2, . 

. . and  0 

s p a 1.  The  random 

variable  Aq 

has 

an  arbitrary  distribution 

but,  if  it  is  taken 

to  be  exponential 

with 

parameter 

X , then 

the  X ' s , n » 1,2, . 
n 

, . . , form  a 

stationary  sequence  of  dependent  exponential  random  variables. 

The  model  of  (1.1)  and  (1.2)  will  be  called  EARMA (1*1) « 
(exponential  autoregressive-moving  average,  each  of  order  1); 
it  extends  the  moving  average  model  of  Lawrence  and  Lewis  (1975), 
which  is  essentially  (1.1)  with  An-1  replaced  by  €n_j,  And 
the  autoregressive  model  of  Gaver  and  Lewis  (1975),  which  is 
just  (1.2),  by  combining  the  two  basic  structures  defined  in 
those  papers.  Further  extensions  are  given  by  Lewis  (1975). 

The  model  defined  by  (1.1)  and  (1.2)  is  actually  a back- 
ward model;  an  equivalent  forward  model  can  be  defined  similarly. 
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However,  the  two,  while  similar,  are  not  equivalent.  This  is 
because  the  sequences  are  not  time  reversible  in  the  sense  that 
{Xj,...,Xj(}  does  not  have  the  same  joint  probability  distribu- 
tion as  {X_^, . . . ,X-k> . The  properties  of  one  model  can  be 
derived  by  the  same  techniques  as  those  of  the  other,  so  we 
consider  only  the  backward  model.  The  time  dependence  does, 
however,  come  in  in  an  essential  way  in  estimation  problems  for 
the  model. 

Let  Tf  = X^  ♦ . . . ♦ X . Then  Tf  can  be  thought  of  as  the 
time  of  occurrence  of  the  r—  event  in  a point  process  having 
(X^ } as  the  interval  sequence.  Further  let  Nt  be  the  number 
of  events  that  occur  in  (0,t]  in  the  point  process. 

Various  moments  and  joint  distributions  for  the  {X^} 
sequence  are  obtained  in  the  next  two  sections.  A recursive 
scheme  for  obtaining  the  Laplace  transform  of  Tr  is  obtained 
in  Section  4.  The  variance  time  curve  and  sequence  are  then 
obtained.  In  Section  6 the  asymptotic  behavior  of  the  sequence 
(Xn>  is  studied  and  limit  theorems  for  Tn,  and  E[Nt) 

are  given. 

Finally,  several  extensions  of  the  model  are  discussed 
as  well  as  another  model  having  similar  correlation  structure. 
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2.  SOME  PRELIMINARY  PROPERTIES  OF  THE  EARMA  (1,1)  MODEL 


Let  (Un)  and  {Vn>  be  independent  sequences  of 

independent  random  variables  taking  the  values  (0,1)  with 

P{U  = 0}  * 0 and  P{V„  = 0}  = p.  Then  we  may  write  (1.1)  and 
n n 

(1.2)  as 


n 


n 


Vn-1' 


(2.1) 


with 


A * PA  ♦ V ( 
n n-i  n n 


(2.2) 


for  n = 1,2,...,  where  (€n>  is  a sequence  of  independent 
exponential  random  variables  with  parameter  X.  Unless  other- 
wise indicated  we  will  assume  that  Aq  * that  is,  Aq  has 

exponential  distribuMon  with  parameter  X and  is  independent 
of  {«n;n*l,2,...},  {UnJ,  and  (Vn). 

It  is  not  hard  to  show  that  Xn  has  an  exponential 
distribution  with  parameter  X;  in  fact  by  (2.2),  for  s a 0 


X+sp, 


E(exp(-sAn)}  * E(exp(-spAn_1)  ) (-^—-) 


for  n « 1,2,...  .By  induction  (Gaver  and  Lewis,  1975), 
E Iexp{-sAn } ) = for  n * 0#1»2,... 


since  AQ  « ( q . Hence 


E{exp(-sXn)}  » E(exp{-s(B€n-tUnAn_1) )) 

‘ B its 

X 

" T* s' 
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showing  that  the  marginal  distribution  of  the  Xn's,  like  those 
of  the  (n'8*  exponential  with  parameter  >. 

However*  the  xn's  ere  not  independent*  as  seen  by  the 
following  calculation  of  the  covariance  between  XR  and 
Ignoring  terms  which  cancel,  we  get 


Cl.l<k)  " E<xnVk>-E(xn>E(x„+k> 


lll-8|,E"nWl'-E|(nIEIWl 


♦ ll*e)’,EIViVk-il  -EIViIEIWilL 


By  induction  arguments  we  get 


and 


E(VW  -E(fn)E,An+k»  * 


E(Vn+k>  -EIAn»E(Vk'  * T- 


Therefore*  the  serial  correlation  p(k)  = ^(kJ/Vartx)  is 


where 


P (k)  » corr(Xn*Xn+k)  = Pk”1c(B,p), 


c(B,P)  - B ( 1—8 ) C 1 — P ) ♦ (1-8) 2p 


(2.3) 

(2.4) 


B - 3Bp  + 28*p  - 82  ♦ p 
B(l-B)  + p(l-B) (1-2B)  . 


When  p * 0 we  have  the  correlation  for  the  EMM  model 
given  by  Lawrence  and  Lewis  (1975) . When  6 ■ 0 we  get  the 
correlation  for  the  autoregressive  model  EAR1  given  by  Gaver 
and  Lewis  (1975).  By  (2.3)  and  (2.4)  the  first  order  correla- 
tion is  nonnegative  and  bounded  above  by  1.  Note  that*  if 
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1-3 


=for 


if 


yg-  c ( 6 # p ) - 1 - 3p  - 23(l-2p)  * 0,  then  8 = yyr 

y « p < y this  value  of  B is  non-positive  so  that,  for  fixed 

p , c(8,p)  decreases  monotonically  from  the  value  p at  6 ■ 0 

to  zero  at  8*1.  If  | < p < 1,  then  * ** 

3 1 

P:  c(B,y)  < 0.  Hence,  for  fixed  p,  as  a function  of  8 
c(8,p),  which  always  is  equal  to  p at  £ * 0 and  equal  to 
0 at  6*1,  is  single  valued  on  [0,1]  for  pay.  For 
p < y it  is  double  valued  on  [0,1] . This  result  will  be  use- 
ful in  the  estimation  of  p and  6.  In  fact  if  p a y esti- 
mates of  p and  6 can  be  obtained  from  the  first  two  serial 
correlations.  For  p < y higher  order  joint  moments  are  needed. 

Note  that  the  second  order  joint  moments  or  correlation 
structure  is  that  of  the  so-called  ARMA  (1,1)  model  (cf.  Box 
and  Jenkins,  1971);  consequently  the  spectrum  will  be 

l “ 

r («)  * £{1  ♦ 2 l p (k)  cos  (ku>)  } 

* " k*l 


. I fl  + (l-2c(6,P)  }p*  - 2p{l-c(6«P)cos  u»n  (2.5) 
v L 1 + 6*  - 2p  cos  w J 

for  Otudi,  The  spectrum  has  the  constant  value  ^ when  the 
Xn's  are  independent,  (p*B=0;  6*1). 

Note  that  the  correlations  p(k)  are  all  positive  in 
this  EARMA  (1,1)  model,  unlike  the  ARMA  (1,1)  model.  This  seems 
to  be  the  greatest  limitation  of  the  model. 
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3.  HIGHER-ORDER  JOINT  MOMENTS 


We  now  proceed  to  the  calculation  of  other  joint  moments 
for  {Xn).  These  are  useful  in  estimating  0,  since  it  is  not 
possible,  when  p < y , to  distinguish  between  the  case  {0,p} 
and  {(1-0), p}  on  the  basis  of  the  second  order  joint  moments. 

This  is  closely  related  to  the  question  of  time  invariance  dis- 
cussed in  the  Introduction;  the  time  (or  serial  number)  depend- 
ence shows  up  clearly  in  higher  order  joint  moments. 

First,  for  k * 1,  eliminating  terms  that  cancel,  we 

have 

C2,l(kl  - B(Xj*n+k)  -E(Xj)E(Xn+|t) 

+ !»ll-»llE"A-lVH|-ElinVlIE(Vw11 

* pk"1|E(«B>Var(«I|)26<l-»)s 

+ (E(f^)  - E(«’)E(‘n)  He’  (1-6)  (1-p)  + (1—6) * p >1 

* p1**1^  (26(1-8)  I + 4(6!  (1-6)  (1-p)  + (l-6)lp».  (3.1) 

When  p - 0,  so  that  we  have  just  the  first  order  moving  average 
process,  (3.1)  becomes  for  k - 1 

C2 , i (1)  * p-(20(l-0)  (1+0)  } = ^y{2  p(l)  (1+0)}. 
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It  is  useful  to  write  the  multiplier  of  p*1”1/!*  in  (3.1) 
as  a polynomial  in  p and  a polynomial  in  6 and  we  then  have 

c2,i(k)  = pk”1  5T  u-SHzea+e)  + pu-b-b2) } 

= pk_1  ^ {p  + B(2-2P)  + B3 (P-2) ) . 

Similarly  we  get  for 

Cl,2(k)  = E(XnXn+k)  -E(Xn)E(Xn+k> 

= Pk_1  jr  ((B(l-B)  (1-p)  ♦ (1-B)2p)(1+B)  (3.2) 

♦ pk_i{p2  + B (l-3p2 ) + B2 (2p*-l) 

, 2 (k-1) 

= r[Cl,l(k) (1+F)  + x7  (p2+?(l-3p2)  + B2 (2p2-l) . (3.3) 

Again  (3.3)  can  also  be  written  as  polynomials  in  B 
and  p.  When  k = 1 we  get 

X J 

^1  2(l)  - (1”B)  (6(2+8)  + p(l+B)  (1-2B)  ♦ p2  (1-2B) ) (3.4) 

* p(l+p)  + B( 2 (1-p) -3p2 ) + 6 2 (-1-P+2P2)  + B*(2p-1) . (3.5) 

The  fact  that  C2  ^ C k)  y C.^  2(k)  indicates  that  the 
(Xn)  sequence  is  not  time  reversible.  This  can  hopefully  be 
exploited,  as  in  the  EMA1  process,  to  estimate  B#  and#  in 
particular,  differentiate  between  the  cases  p#0  and  P, (1-0). 

Higher  order  moments,  e.g.  C2  2(k),  are  useful  in  deriving  the 
asymptotic  variances  of  sample  serial  correlations  for  the 
model.  This  will  be  discussed  elsewhere. 


The  above  third-order  joint  moments  are  special  cases 
of  the  third  order  joint  moments  with  two  lags,  j and  k, 
whose  double  Fourier  transform  will  give  the  bi spectrum  of  the 
{Xn>  sequence. 

For  j a 1,  k * 1,  and  p < 1,  similar  calculations 
to  those  above  show  that 

Cl,l,l(J'k>  * E<XnWn*j+k>  -E<VE(Xn+j)E<Wk> 

- {0(l-0)2pkp2(j“1)  (l-p)  + (1-0)  *pkp2^}  (3.6) 

♦ (0  (1-0) 2 (pk{p2j  + 1. 

+ 2p2j“1(l-p)  + 2pj“1(l-p) (l-^j“1)+p^} 

+ pk-1{  (l-p)Jpj’1+  (l-p) } 

+ pj"X{(l-p)  ll-pk“1] ♦ P) J 

+ 02(l-0)  (pk(pj'1(l-p)J  «■  pk-1  (1-P)  +Pj’1(l-P)J 
+ (l-0)5(pk{±^+2Pj(l-Pj)> 

+ pk’1{(l-p)pj  + Pj(l-Pk"1))]}. 

We  give  this  expression  for  completeness  and  because  it 
is  clear  that  second-order  joint  moments  do  not  describe  com- 
pletely a process  which  is  as  non-normal  as  the  EARMA(1»1) 
described  here.  It  is  felt,  however,  that  the  special  cases 

(3.1)  and  (3.2)  of  the  third  order  moment  C.  . .(j,k)  when 

1,1,1 

j - 0 and  k * 0 respectively  will  give  all  necessary  information. 
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It  is  also  possible  to  derive  Laplace  Stieltjes  transforms 
for  the  joint  distributions  of  several  X^'s.  These  will  be 
multivariate  exponential  distributions.  Thus  we  have , for 
example, 


E[exP<-»1V,2xn+l)1  * 


r,  A j 

r.  x \ 

iHA+Bs^)  X (1—0)  (X+Bs^+pSj)* 

A+s^B 

*+s2BJ 

(X+S1)  1 (X+S2P) (X+Sj+SjP) (A+Bj^+Sj) 

0.7) 


Despite  its  relatively  simple  form,  this  transform  does 
not  lend  itself  to  easy  derivation  of  moments,  e.g.  ElX*Xn+iI, 

E^XnXn+l^#  or  E*XnXn+ll*  or  of  conditional  moments,  as  for 
the  EMA1  case  (p  * 0)  in  Lawrance  and  Lewis  (1975);  nor  is  it 
invertible  to  give  approximate  likelihoods  for  the  process. 

When  8i  * 8 2 * 8 we  9et  the  transform  of  the  sum 


Elexp{-s(Xn+Xn+1) }] 


»(>*Bll  . X (1-6) (A+fl.+p.) 

(A+s)  ( A+ps ) { A+s ( i+p ) } ( A+s+sB ) 


(3.8) 


Transforms  of  the  distributions  of  the  sums  of  adjacent 
X^'s  are  very  useful  in  the  point  process  theory  of  the  model; 
if  these  can  be  obtained  one  can  obtain  the  second  order  proper- 
ties of  the  counting  function  of  the  point  process  in  the  form 
of  either  an  intensity  function,  the  (Bartlett)  spectrum  of 
counts  or  the  variance  time  curve  (see  Cox  and  Lewis,  1966, 

Ch.  4).  These  transforms  are  discussed  in  the  next  section. 
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4 . THE  SEQUENCE 


(Tr). 

Recall  that  Tf  ■ ♦ . . . + xr . We  will  interpret  Tr 

as  the  time  of  the  r—  event  in  a point  process  which  starts 
with  an  event  at  the  origin.  We  will  obtain  a recursive  rela- 
tionship for  computing  E(exp(-sTr)}  for  r = 1,2,...  and 
s a 0.  Let 


<Ms1#s2)  * Elexpf-s^  - SjA^l . (4.1) 

Then,  by  direct  computation  from  the  definition  (1.1)  and  (1.2) 

{X  (X+Sj^+s-jP)  )2 

'|>(s1,s2)  ■ (X'+SjB+Sj)  "(X+Sjfl)  (A<fs2P)  (x+Sjp+Sj^)  * «»2) 

Now  we  define 


blB1,*2)  « Efexpf-tSj^B+SjVj^)^}]  (4.3) 

A (A+s^B+s2p) 

* (X+s^+SjHV+i^) 


and  let 

♦r<*i»a2)  * E (expt-s^  - s2Ar)] . 

Then,  for  r a 2 

^r(sl#a2)  * ElexP^’Bi<Tr-i'*‘e(r‘fUrAr-i)  " 82(pAr-l+Vr€r)  ^ 

« E{exp{-(s1B+s2vr)  «r  - s^^j^  - (s1Ur+s2p)Ar_1)l 


For  B * 0 and  P = 0 we  get,  respectively,  the 
recursion  relationships  for  the  EAR1  process  (Gaver  and  Lewis, 
1975)  and  the  EMA1  process  (Lawrence  and  Lewis,  1975)  which 
have  explicit  solutions  which  lead  to  expressions  for 
E{exp(-sTr)  ). 

Using  (4.4)  we  can  calculate  the  Laplace  transform  of 
Tr  recursively.  In  particular  we  have  for  that,  using 

(4.1) 


♦ (s,0)  * E [exp{-sT^} ] « , 


(4.5) 


as  it  should.  Then  we  get 


4>2 (s,0)  * E (exp(-sT2 } ] 

■ (1*8)  <x-»-se)  n.»p.»)](4,<l 

which  agrees  with  (3.8).  Unfortunately  the  expressions  become 
very  unwieldy  as  r increases.  We  have 

V*'01  * [f’rr?* 6 a~el { (U»+«p) 

. (X  (X+sB+sp*)  }* 

(X+sB)  (X+sB+s)  (X+sp1)  (X+s+sp*) 


For  the  transform  of  the  distributions  of  Tj  and  T5  we  get 
♦4  <s,0)  - [S*  5^+  62  (1“B>kJ1(X+sB)  (X+sB+s*(X*sp^J  (X+s+spty 
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'”r 


4 8(1 


A4S84BP  v r (If  A-f  sB^sp^-f  s l } 2 - 

’6)  jjl+»8)  (X+8B+8)  {A4a84B4Sp)  X ^ (A+SP^SP^*1)  (X+8+8pk+8pk+1) 


(A4aB4ap2) 


(A4sB)  (A+sB+s)  (A4a84s4ap 


{ A ( A4s84sp4ap 1 ) ) 2 

7T  (A+8P+8P*)  (A+a+ap+ap-*; 


, ..  Q.i  (A4bB4bp)  (A4sB4ap4sp2 ) (A  (A+bB+8P+8P2+8P 3 ) )* 

(A+bB) (A+aP+a) (A+aB+s+spi (A48&4848P48P* ) 


IT 


+ 8P+8p'f+8p  b ) (A+8+8p+8p  J+SP  1 * (<*8) 


For  the  sura  we  qet  the  transform 

’/'s (s#0)  - {e4  J 


[ A ( A+sB+sp^n  2 


4 B' 


(1’B)2[j 


k«l  (A+6B) (A+aB+a) (A+ap  ) (A+a+ap  ) 

< ( A-faBtsp^"1)  [A  (A+aB+ap+sp*) ) 1 
2 ’ (A+aB)  (A+bB+b)  (A+aB+a+ap*1"^) 


lr  Jr 

(Atap+ap  ) (A+a+ap+ap 


r> 

+ * ( (A»aB-f-ap^~2)  (A (A-faB-*-ap2<fapk)  1 2 

k-3  ’ (A+aB)  (A+sB-f-a)  (A+aB+a+ap  ” ) (A+ap2+ap  ) 

* 1 

A + a 4 ap2  4 sp  1 


. (A4aB4ap)  [A  (A4sB4SPi48P<>)  1 2 

(A4aB)  (A4aB48)  (A+aB+a+ap)  (A4ap ’4-ap') 


r+ 


1 _~1 
a + ap  * 4 apM 


4 B (1 


-e)*[- 


A 4 aB  4 80  , 

(A4bB)  ( A 4s B 4s ) ( A4sB+s+8p) ( A4aB+s4ap4sp; ) 
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< (X+sB+SP+SP2  ) 

( (A+SP  + 8P,+8P1)  (A+s+BP+SP^+SP  *) 

(X+sB+SP+Sp2)  [X  (A»gB»lpa-»«P>4-»pl,)1a 
(A  + 8P,+8P,+SP1’)  (X+a+SP4+SP*+SpM 

(X+sB+SP2+SP  3 ) [X  ( X+sB+SP+SP *+SP*)  ] 2 \ 

(A  + SP  +8P  ,+SpJ' ) (X+S+Sp  +Spi+8P*)  f 

(X+sB+Sp2  ) (X  + sg+SP+SP  * ) (X  (X+sB+SP+SP 2 +8P%)  ] % 

(X+sB)  (X-faB+s)  (X-fsB+s+sp2^  (X+BiJ+s+sp+sp1) 

x 1 1 

(X  + SP  + SP^SP*1)  (X+S+Sp+Sp'+Bp^J 

, ,,  ot»(  (X+sB+sp)  (X+sB+sp+sp2)  ( X+sB+sp+sp 2 +sp 2 ) 

I (X+sB)  (X+sB+S)  {X  + sB+S+SPnX+S&+8+SP+BpM 

x [X  (X+sB+sp+sp^sp^sp**)  ] 2 

(X+sB+s+sp+sp'+sp*) (X+sp+sp4+sp  *+sp^ ) 

* X + 8 + sp  + sp2  + SP*  + sp*  } • 

The  pattern  in  these  results  is  fairly  evident,  but  it 
is  clear  that  the  Laplace  transform  of  the  intensity  function 
(Cox  and  Lewis,  1966,  Ch.  5), 

* 00 

mf  (s)  = £ i|»_  (s;0) 

1 r=l  r 

is  not  obtainable.  This  is  disappointing  in  view  of  the  simpli- 
city  of  the  result  for  the  EMAl  process  (Lawrance  and  Lewis,  1975). 
It  is  probably  true  that  as  much  information  can  be  obtained 
from  the  higher  order  joint  moments  given  in  Section  3 as  can 
be  obtained  from  the  intensity  function.  As  a particular  case. 


13 


for  the  EMA1  process,  the  intensity  function  differentiates 
the  cases  8 and  1 - 0,  where  the  serial  correlations  do  not. 
However,  a direct  estimate  of  0 is  obtainable  from  2(1) 
and  C2  ^(1)  which  are  given  at  (3.1)  and  (3.2). 

Some  idea  of  the  behaviour  of  the  intensity  function  can, 
however,  be  obtained.  The  limiting  behaviour  is  discussed  in 
Section  6.  Consider  now  the  value  of  the  intensity  function 
at  0. 

Note  that 


pr(Xx  ♦ ...  ♦ Xn*  t)  * ...  + <n<|) 

. ; litiLL 

Le  \Ti 


(4.10) 


Since  X^  is  exponentially  distributed,  we  have 


-It, 


pr(Ntal)  - prtX^t)  » (1-e  AW) 


and,  from  (4.10) 


l pr(N.  ik) 
t 


s 


\ 

0 


*) 


t 


A , (e  et-l) 

0 t ' 


which  tends  to  0 as  t -*•  0. 

Now  the  intensity  function  is  the  derivative  of  the 
function  E(Nfc). 
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Hence,  we  have 


EOT) 
lira  — r-S- 
t*0  c 


lim 

t*0 


l pr(N  ik) 
k«l  * 


t 


1 - e”** 

lim  i — f « Xj 

t+0  c 


i.e.,  that  the  derivative  of  the  mean  value  function  (the  intensity 
function)  at  t * 0 is  X. 


5.  THE  VARIANCE -TIME  CURVE  AND  SEQUENCE 


We  can  easily  obtain  the  variance  function  for  Tr  and 
the  index  of  dispersion  for  intervals  (Cox  and  Lewis,  1966, 
p.  71),  from  the  results  of  the  previous  sections. 

For  the  variance  function  we  have 


Var(Tr) 


r-1 

r Var  (X. ) +2  l 
1 i-1 


(r-i)(E(X1Xi+1)  - E(X1)E(Xi+1)) 


1 

T* 


r-1 

(r  + 2 l 
i-1 


(r-i) ((l-B)pi“1(B(l-P)  + (l-B)p)) ) 


1 

X* 


r-1 

(r  + 2r (1-B) (B(l-p)  ♦ (1-6) p J l 

k-1 


P 


i-1 


r-1 

2(1-6) [B(l-P)  ♦ (1-B)P]  l 

i-1 


} 


1 


r-2  . r-1 

(r  ♦ 2(1-6)  { 6 ( 1— p ) + (1-B)  p Hr  l PJ  - l 

j-0  i-1 


iP1’1)) 


If  P < 1 

Var(Tr)  - yr  (r  ♦ 2(1-6)  (B(l-p)  ♦ (l-B)pHr  ^ 

j* 

- jt  lr  ♦ 2(1-6)  ( 6 ( 1— p ) + (1  3)p)(I^-  )).  (5.1) 


Hence,  for  p < 1 the  normalized  variance  sequence  is 

Var(T  ) r 

Jr  " FFTT^T7  " 1 + 2(1-6)(B(1-P)  ♦ (l-6)p)(y~-  p-ff-pF*  • 

Therefore,  for  p < 1 the  index  of  dispersion  is 

J - lim  Jr  - 1 ♦ 2(1-6) (6 (1-p)  ♦ (1—6) p ) . (5.2) 
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For  p » 1 


Hence 


Var(Tr) 


prlr  + 2 (1-3 ) 2 


r-1 

l 


i=l 


(r-i)l. 


lim 

n-*» 


Var {Tf } 

FeTt^P 


lim  1 + 2 (1-6)  2 

f-MO 


r-1 

l k 


k-1 

r 


Thus  the  process  is  overdispersed  relative  to  the  Poisson 
process  for  which  lim  J * 1.  In  particular,  if  an  observed 

r-~>  r 

process  had  exponentially  distributed  marginal  distributions 

for  the  X.  but  lim  J was  much  greater  than  one,  the  EAKMA 
1 r4®  r 

(1,1)  process  could  be  a candidate  as  a model. 

As  a byproduct  of  (5.2),  for  p < 1 we  get  values  for 

the  slope  of  the  variance  time  curve,  lim  V'(t)  ■ V'(*)#  the 

t-*» 

initial  points  of  the  spectrum  of  counts  q+(w)  and  the  spectrum 
of  intervals  f+(w)  as 

f+(0+)  = ? lim  Jr * (5.3) 

r-^*> 

V’  (<*>)  = x lim  J , (5.4) 

j--.cn  1 

g+(0+)  = ^ lim  J , (5.5) 

r+oo 

for  p < 1,  (Cox  and  Lewis,  1966,  4.6.12). 

The  relationship  f+(0+)  can  be  verified  directly  from 
(2.5);  (5.5)  says,  using  (5.2),  that  the  initial  point  of  the 
spectrum  g+(w)  is  greater  than  its  value  ^ for  a Poisson 
process. 
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6.  LIMIT  THEOREMS 


In  this  section  we  will  study  the  mixing  properties  of 
{xn>,  obtain  strong  laws  and  central  limit  theorems  for  {TR> 
and  (N^ } , and  obtain  renewal  type  results  for  EfN^].  We  will 
assume  throughout  this  section  that  p < 1.  Our  technique  usually 
will  be  to  use  the  limiting  behavior  of  related  Markov  chains  and 
Markov  renewal  processes  to  obtain  limiting  results  for  the 
EARMA  (1,1)  process.  The  Markov  chain  and  Markov  renewal  process 
will  have  the  real  line  as  state  space.  Although  it  is  usually 
hard  to  obtain  specific  results  concerning  the  limiting  behavior 
of  these  processes  if  they  have  a nondiscrete  state  space,  the 
linear  forms  of  (1.1)  and  (1.2)  allow  us  to  obtain  the  limiting 
behavior  quite  easily  in  this  instance. 

6.1  Mixing  properties  and  asymptotic  normality 

We  first  consider  the  limiting  distribution  of  the  sequence 

(Xn)  in  the  case  when  AQ  is  not  exponential  (X).  From  (1.1)  it 

follows  that  the  dependence  of  Xn  on  Xn-1  is  through  the  auto- 

regressive  sequence  { An ) . Since  An  = pAn-1 + vn*n  (n*l,2,...), 

(An;  n*0,l,...}  is  a Markov  chain  (a  discrete  time  parameter 

process  which  satisfies  the  first-order  Markov  property),  with 

state  space  (**+#B+)  # where  R+  = (0,®)  and  denotes  the 

collection  of  Borel  subsets  of  R+.  The  transition  function 

P(x,B)  * pr(A_  t B I A.  * = x)  for  the  Markov  chain  is  given  by 
n n— l 

P(x,B)  - pc  (B)  + (1-p)  f Xe"Xy  dy  (6.1) 

P * BJpx 

for  any  interval  B of  and  x & 0,  where 
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p 


1 if  pxeB, 

0 otherwise 

and  for  z 0,  B-z  = (y-z:y€B). 


c (B)  = 
px 


Thus,  An  has,  conditionally,  a shifted  exponential  distribution 
with  an  atom  at  the  point  px.  By  (1.2)  we  have  the  expansion 

\ - °\  * pk'1  Vi + • • • - pvk-iVi + Vk- 

and  it  is  not  hard  to  show  that 

E(exp{-s(Vk(k  ♦PVk_l«k.1 + ...  + Pk"1V1€1)  } ] * ^f£-, 

* pk+  a“pR)I?s* 

Hence,  for  the  k—  order  transition  function  we  have 


Pk(x,B)  = pr ( A.  >-  B | Aft  * x) 


= Pke  k (B)  + <l-Pk)eAp 
P x 


'xl 


\e~^  dy 


(6.2) 


B 


for  k £ 1,  x i 0,  and  any  Borel  set  B in  R+.  Taking  limits 
as  k -*  ® we  have 


lim  Pk(x,B)  = *(B)  5 [ *e”*^dy  , 


B 

which  is  simply  an  exponential  distribution. 

From  (1.1),  adding  on  the  moving  average  operation  to  get 
the  complete  (Xn>  process 

P ( Xn  € B IAq-x)  * ',pr(8<ncB}f  (l-B)pr(B<n  ♦ An-1  « B | AQ»  x) 

= f.Ti  (Bp”1)  ♦ (1-B)  | 7T  (dy)Pn-1  (x,B-By) . 
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where  Bg"1  * {yg”1:ycB}.  Hence,  taking  limits 

lim  P(X  € B | Afl  - x)  - BttJBB-1)  + (1-6)  f * (dy)  it  (B-By) 

n-*o»  u 1 

- ti(B). 

Thus  we  have  shown  that  if  we  start  generating  the  sequence  (Xn> 

with  a random  variable  AQ  that  has  an  arbitrary  (and  possibly 

degenerate)  distribution,  the  distribution  function  of  XR  will 

converge  to  the  exponential  distribution  as  n -*• 

We  will  now  study  the  mixing  properties  of  (Xn>  when  Aq 

is  exponentially  distributed.  Let  be  the  o -algebra  generated 

by  the  random  variables  A-,...,A  . and  U.  , ...,U_. 

u in  i in  X RK 

Let  be  the  o-algebra  generated  by 

'm-'m+l'"-’  and  1x51  'Em*  • (respectively  L1  (^1 ) , 

be  the  collection  of  real-valued  functions  that  are  measurable 
with  respect  to  £m,  (respectively  gm),  and  have  finite  second 
moments.  Since  (<n)  and  (Un>  are  independent  sequences  of 
random  variables,  by  the  Markov  property  of  (AR}  we  have  for 

f e (£m>  and  9 e k * l*  that 

E(fg)  -E(f)E(g)  • E (f {E (g  | Am)  - E(g)  }] 

- Eif  J E(g  | Am+k-yHPk<vdy) 

« pkE(f  f E(g  | Ara+k-y){e  R (dy)  - it  (dy)  ) J 

p Am 

k ( xPkA™ 

♦ (l-pK)E{f  j E(g  | Am+k-y)  (e  m-l)ii(dy)}  (6.3) 

by  (6.2). 
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By  Holder's  inequality 

Etf  I E(g  | AB+k  = y)c^^  (dy) ) 

- EifEi’iL-»‘v' 

* E<fJ)l/JE[{E(|g|  I *m+k-p\)>,l1/2. 

* E(f2)1/2E{E(g2  I Am+k=  PkAm)  )1/2. 

But#  letting  z » pky 

E(E<9*  I W^V* 

= j Ae"XyE(g2  | Am+Jt « pky)dy 
0 

, Jj.  | ie--»  E (ga  | A^-.ld, 

0 

* -T7  E(g2), 

P 

if  0 < p < 1. 

Thus,  for  0 & p .<  1 

pkE[f  f E(g  | A *=  y)  U . (dy)  - *(dy))] 

P Am 
in 

k 

* PJ  E(f2)1/2E(g*)1/2  - pkE(f)E(g).  (6.4) 
Using  the  H61der  inequality  again  we  have 

E{f  IE(g  * (eApkAm- l)"<dy)} 

« E(g)E{f(eXp  A»  - 1) ) 

* E(g)E(f2)1/2E((eAp  ^ - 1) * )1/2. 
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For  k sufficiently  large , and  0 < p < 1 


E(  <e*p**m  - 1) 1 } « E(e2XpAAm- 2eXpkAm+ 1) 


X (l+2p  j 


-2 


X(l+pk) 


+ 1 


< 2p 


1 + pk  - 2(l+2pk)  + (l+2pk) (l+pk) 
(l+2pk) (l+pk) 

2p2k 

TE 2Tc 

1 + 3p*  + 2p*K 

2k 


Hence  from  (6.3)  and  (6.4) 


k 

E(fg)  - E(f)E(g)  * E (f 2 ) 1/2E  (g2 ) 1/2  (p7+pk+/I  pk) 

k 

« E(f2)1/2E(g2)1/2p?(2+/J).  (6.5) 

Therefore,  when  AQ  is  exponentially  distributed  the 

sequence  {(A  . € ,u  )}  is  strong  mixing  in  the  sense  of  Rosenblatt 
n n n 

(1971)  and  is  in  fact  asymptotically  uncorrelated  in  the  sense  of 
Rosenblatt  (1971).  By  (1.1)  {Xn>  is  also  strong  mixing  and 
asymptotically  uncorrelated.  Further,  for  any  measurable  sets 
B * f(X1,...,X|n)  and  C ■ 9<xm+jc»xin+)c+]/*'  •)  for  suitable  func- 
tions f and  g,  by  (6.5) 

k 

|pr <B  fi C)  - pr  (B)pr  (C)  | « (2+/2)p^  . (6.6) 


In  addition  it  follows  that 


lim  ^ 


n 


1 

l 


a. s. 


(6.7) 


by  the  strong  law  for  a stationary  sequence  of  random  variables; 
cf.  Doob  (1953,  Chapter  10).  Also  since 
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(6.8) 


we  have 


lNt+l 

“n 


t 

lim  -r-  * X a.s. . 

t-M.  z 


(6.9) 


To  obtain  a central  limit  theorem  for  (Tn)  we  note  that, 
using  (5.2),  the  hypothesis  of  Theorem  (20.1)  in  Billingsley  (1968) 
is  satisfied  by  (6.6)  with 


o 


2 


Var  X.  + 2 l cov(X. ,X.  ) 

1 n-1  1 i+n 

X (1  ♦2(l-B)(B(l-p)  + (1-0)  p)  , 


(6.10) 


which  is  positive.  Hence, 


where 


,T  -nf-  t 

lim  pr| * x>  * 4>(x), 

n-M»  '/no 


- 5 r*  ~ jy2 

♦(x)  - <2*)  * I e * dy. 


(6.11) 


By  the  remark  after  the  proof  of  Theorem  (17.3)  in  Billingsley 
(1968)  we  also  have 

( Nt"Xt  \ 

pr»^W75  • x)  •♦«*>• 


(6.12) 


i.e.,  the  counting  process,  suitably  normalized  is,  in  the  limit, 
asymptotically  normally  distributed. 

- Tn 

Equations  (6.7)  and  (6.11)  show  that  the  estimate  X * ~ 
of  E(Xn)  is  strongly  consistent  and  asymptotically  normally 
distributed  with  variance  ^o2,  with  o2  defined  by  (6.10).  Also 


23 


of  X is  also 


* t 

(6.9)  and  (6.12)  show  that  the  estimate  X * ~ 

strongly  consistent  and  asymptotically  normally  distributed  with 
n * X * 

variance  — ^ — . 

6.2  Renewal  type  theorems 

We  will  now  turn  our  attention  to  renewal  type  limit  theorems 
for  E(Nt)  as  t ■*  «. 

For  any  Borel  set  B in  R+,  t * 0,  and  x a 0,  let 

Q(x,B,t)  = pr(Ax  <B,  Tx<t  | AQ*x)  . (6.13) 

From  the  definition  of  the  process  (1.1)  and  (1.2)  we  have 

Q(x,B,t)  - p ex(B)  [eU“exp(-X6'1t)} 

+ (1-6)  ll-expf-AB^maxU-x,*))))) 

♦ (1-p)  | Xe“Xy  dy  Ifi(px+y) (61  |0>tj (By) 

0 

+ (1-B) I | tj (By+x) }, 

where 

i 1 if  z f B, 

v*>  - 

' 0 otherwise. 

By  (1.1)  and  (1.2),  ( (An*Tn) ;n * 0,1, . . . ) is  a Markov 

renewal  process  in  the  sense  of  £inlar  (1975,  Ch.  10),  with  state 
space  (R+»g+)  and  semi-Markov  kernel  Q defined  in  (6.13); 
that  is,  letting  TQ  « 0, 

pr *An+l € B#Tn+l  “ Tn  * t I A0'  * * * ,An'T0' * * * 'Tn*  (6*10 

- Q(An,B,t) . 
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Thus  the  TR  process  of  the  EARMA1  process  is  a marginal  procass 


in  a Markov 

renewal  process  with  a non-discrete  state  space. 

Let 

where 

CD 

R(x,B,t)  = l Qn(x,B,t) , 
n=0 

(6.15) 

Q°(x,B,t)  = Ex(B)eQ(  (0,t)) 

(6.16) 

and 

Qn(x,B,t)  = f f Q(x,dy,ds)Qn"1(y,B,t-s) 

J J 

0 0 

for  n a 1, 

x a 0,  t a 0,  and  Borel  set  B . Note  that 

E (Nfc  | AQ  = x)  « R(x,E,t)  . 

(6.17) 

Since  {*n}  is  a sequence  of  independent  exponentially 
distributed  random  variables,  for  any  Borel  set  B with  positive 
Lebesque  measure 

pr(An«B  infinitely  often  | AQ  • x)  ■ 1 

for  all  x * 0.  Therefore,  condition  A of  Jacod  (1971,  p.  86) 
is  satisfied.  Further,  sinc^ 

Pr  (An  c B,  Tn*t  | Aq  = x) 

a Bnpn”1  (l-p)pr(c  ( B - pnx, €.  + . . .+€  * t0~*) , 

n in 

for  n a 2,  a > 0,  and  each  6 > 0,  there  is  a Borel  set  C 
of  R+  of  positive  Lebesgue  measure  so  that  the  density  of 
Qn (x, . , (a-6,a+6))  with  respect  to  Lebesgue  measure  has  a positive 
lower  bound  on  the  product  set  Cxc.  Hence,  the  hypotheses  of 
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Jacod  (1971,  Th.  3,  p.  107)  are  satisfied  and 


lim  R(x,B,t+C) 

t-.<» 


lim  f R(x,B, t+du)  = — tt  (B)  | C | 

t->oo  J m 

c c 


(6.18) 


for  any  Borel  sets  B and  C of  R+  and  x a 0,  where  |c| 
denotes  the  Lebesgue  measure  of  C,  and 

fUD  f oo 

m = J tt  (dy ) j (1  - Q (y ,R+ ,s) )ds 

0 0 

#a> 

* I pr(T1>s)ds  = j-  . (6.19) 

0 

Finally  taking  C - [0,h]  and  B * R+  in  (6.18)  we  have 


lim(E(Nt+h  | Aq=x)  - E(Nt  | Aq=x)]  » Xh, 


(6.20) 


for  all  x a 0.  This  is  a Blackwell -type  renewal  theorem  for  the 
EARMA (1,1)  process. 

6.3  The  intensity  function 

We  will  now  show  that  the  intensity  function  for  the  counting 
process  exists  and  its  limit  as  t -*•  00  is  X.  Let 

q (x, t)  * lim  r {Q(x,R.,t+h)  -Q(x,R.,t)>  (6.21) 

h+0  n 

j X expf-Xg'^t}  for  t < x, 

( X exp{-X0-*t}  ♦ (1-0)X0”1  exp(-X0”* (t-x) } for  tax. 


Consider  the  Markov  renewal  equation,  (MRE) , 


f (x,t) 


q(x,t) 


0 0 


Q(x,dy,ds)f (y,t-s) . 


(6.22) 


26 


Since  q(x,t)  * 1 and  sup  Q(x,R.,t)  < 1 for  t > 0,  by 

x T 

the  results  of  £inlar  (1975,  Ch.  10,  Sec.  3),  (6.22)  has  the 
unique  solution 


m(x,t)  =|  j R(x,dy ,ds)q  (y,t-s) . 

0 V 

The  function  U(t)  = m(u)du  satisfies  the  MRE, 

0 

f(x,t)  * Q(x,R+,t)  + J J Q(x,dy,ds)f (y,t-s) 

0 0 

which  has  as  its  unique  solution  R(x,R+,t)  -1,  where  RfXfR^t) 

is  defined  by  (6.15).  Therefore,  m(x,t)  is  the  intensity  of 

the  counting  process  evaluated  at  t given  that  A.  ■ x. 

•00  W 

Note  that  J n(dx)q(x,t)  * le”  t,  and  for  each  x e R+ 

0 

t-»q(x,t)  is  monotone  decreasing.  Hence,  for  b > 0. 

CO  #00 

= b l Ti(dx)sup{q(x,t)  j nb  * t < (n+l)b) 
n * 


*b  7T  (dx)q  (x,0)  + b ir(dx) 


j ir(dx)  | q (x, 


t)dt  < • 


Hence,  q is  directly  Riemann  integrable  with  respect  to  * 
in  the  sense  of  Cinlar  (1969,  p.  388).  By  (6.18),  the  hypotheses 
of  the  Theorem  on  page  390  of  £ir.lar  (1969)  are  satisfied,  and  we 
have 


lim  m(x,t)  = >. 

for  all  x a 0.  Thus  the  intensity  function  goes  to  X as  t ♦ •. 

The  same  types  of  techniques,  i.e.  writing  a Harkov  renewal 
equation,  writing  down  its  solution,  and  computing  its  limit  as 
t ♦ ®,  can  be  used  to  show  that  the  distribution  of  the  forward 
recurrence  time  is  in  the  limit  as  t ♦ • exponential  with 
parameter  X. 
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Note  that  subsequent  intervals  are,  in  the  limit,  not 
exponentially  distributed.  Following  Cox  and  Lewis  (1966,  Ch.  4) 
call  these  intervals  L^,L2,...  • Their  marginal  and  joint 
distributions  can,  in  theory,  be  obtained  from  results  of  Section 
3 using  Palm-Khinchine  relationships.  We  note  only  that  their 
means  are,  (Cox  and  Lewis,  1966,  Ch.  4), 

E(LiJ  = E(Xnl(l  + C2(Xn)p(i)},  (i  - 1,2,  — ) . 

1 2 

Since  E[X  J * y and  C (X  ) = 1,  using  the  expressions  (2.3) 

li  A 11 

and  (2.4)  for  p(i)  we  get 

E[L.]  * X {1  + 

This  decays  geometrically  to  j-.  When  p * 0 and  we  have  only 
the  two-dependent  EMA1  process,  the  bias  does  not  extend  beyond 
E[L^I,  as  would  be  expected  from  the  construction  of  the  process. 
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7.  EXTENSIONS  AND  CONCLUSIONS 


The  EARMA  (1,1)  process  described  here  is  a generalisation 
of  the  EAR1  and  EMA1  processes  and  has  the  correlation  structure 
of  a mixed-moving  average-autoregressive  process  of  order  (1,1) 
and  an  exponential  marginal  distribution.  It  thus  provides  an 
alternative  to  an  i.i.d.  exponential  sequence  or  a Poisson  process 
which  is  relatively  simple  probabilistically  and  has  an  easily 
visualized  dependency  structure.  Moreover,  it  is  easy  to  generate 
on  a computer  in  simulations,  and  asymptotic  properties  follow 
easily  from  the  observation  that  it  is  the  marginal  process  in  a 
Markov  renewal  process  with  a nondiscrete  state  space. 

Lewis  (1975)  discusses  methods  of  generating  similar  pro- 
cesses in  which  either  (i)  the  marginal  distribution  of  the  X^'s 
is  Gamma,  mixed  exponential  or  Weibull;  and/or  (ii)  the  dependency 
structure  is  that  of  higher  order  ARMA  processes.  He  do  not  give 
details  here;  probabilistic  properties  of  these  processes*  as 
well  as  multivariate  processes,  will  be  discussed  in  later  papers. 

Similarly  estimation  properties  are  discussed  elsewhere; 
we  note  only  that  results  such  as  those  in  Section  3 allow  one 
to  compute  asymptotic  variances  of  serial  correlations  and  other 
statistics  based  on  the  x^'s.  Thus,  we  have  a tractable  process 
with  which  to  examine  the  efficacy  of  the  many  tests  for  renewal 
and  Poisson  processes  given  in  Cox  and  Lewis  (1966,  Ch.  6). 

A limitation  of  the  process  is  that  the  serial  correlations 
are  all  positive  and,  for  p « 0,  i.e.  the  EMA1  process,  are 
bounded  above  by  1/4. 
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Thus,  the  question  arises  as  to  whether  there  are  processes 
with  exponential  marginal  distributions  and  ARMA  (1,1)  second 
order  correlation  structure  and  which  cover  a broader  range  than 
the  EARMA  (1,1)  process,  though  perhaps  at  a cost  of  more  com- 
plicated structure. 

One  such  process  is  now  discussed  briefly  (Cox  and  Lewis, 
1966,  pp.  7,  194-204).  In  this  two  state  semi-Markov  model  there 
are  two  types  of  intervals  with  p.d.f.s.  f ^ (x)  and  f2(x) 
sampled  in  accordance  with  a two-state  Markov  chain  for  which 


p - 

al 

l-«ll 

1 (7.1) 

and 

f 1-f>2  1“al  1 

n - n P“  2.tt  » y.y=5" 

1-a- 

1 

l*  a2  t a^j 

It  is  assumed  thit  no  knowledge  is  available  about  the 
type  of  interval.  Then,  the  distribution  of  an  interval  is 

fx(x)  ■ w1f1(x)  ♦ *2f2(x)  (7.3) 

and  the  correlation  between  X^  and  X)^  is 

P (k)  » A Bk  (k  ■ 1,2,...),  (7.4) 


where  A is  a positive  constant  and  6 * ♦ a2  - 1 * - (l-a2) . 

Thus  the  correlation  structure  is  that  of  an  ARMA  (1,1)  process. 
If  we  let 


fx(x) 


f2(x) 


I 

I 


>e~)lc 

l-e'Xx0 

0 


0 < X * Xg, 
X > Xgl 
X * Xg, 

X > Xg) 
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the  marginal  distribution  of  an  interval  X,  fg(x),  is 
exponential  (A)  if  we  set  * l-exp(-  x ) . There  is  then 
only  one  degree  of  freedom  left  in  the  matrix  P , and  in  addi- 
tion to  A,  we  have  free  parameters  (or  Xq)  and 

What  then  is  the  range  of  8,  and  can  it  be  negative? 

Straightforward  manipulation  shows  that 


which  lies  between  zero  and  one.  Thus  the  model  is  no  broader 
than  the  EARMA  (1,1)  model  and  the  question  of  obtaining  negative 
correlation  is  still  open.  More  complicated  non-linear  schemes 
for  doing  this  are  discussed  by  Lewis  (1975). 
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